Pioneering new frontiers in AI
Over the past decade, artificial intelligence and data science have undergone a transformative shift, largely propelled by advancements in deep learning techniques. High-dimensional learning tasks, such as protein structure prediction and natural language processing, have proven feasible with sufficient data, although the number of data samples needed to achieve reliable results generally increases exponentially with dimension – a phenomenon known as the curse of dimensionality. These breakthroughs can be attributed to the fact that most relevant tasks are not generic but have inherent regularities due to the underlying
low-dimensionality and structure of the physical world. Revealing such regularities through geometric concepts not only facilitates the study of state-of-the-art architectures but also provides a blueprint for deriving novel ones that adhere to prior (physical) knowledge. In this area of geometric deep learning, neural designs targeting graph-structured data represent a prominent example. Unlike images, graphs do not have a regular structure: nodes may have a varying number of neighbors and are not sampled from a regular grid. Furthermore, a key property of graphs is that the ordering of nodes is usually assumed to be arbitrary, and thus any function acting on graphs should not depend on it. Geometrically speaking, the set of all reorderings, called the permutation group, makes up the symmetries of graphs, and we are interested in neural networks for which the output is either invari-ant for global, graph-wise prediction or equivariant for node-wise ones, meaning that the ordering in the output is tied to the ordering of the input.
Graph neural networks have found widespread applications and have in particular become the de facto standard for transduction in deep learning. Other than inductive learning, which tries to infer a general model from labeled examples in order to predict labels of unseen ones, transductive approaches learn labels simultaneously on training and test data. Transduction therefore avoids solving a more general problem as an intermediate step and thus faces a potentially simpler problem as compared to inductive learning. At ZIB, we derived novel transductive learning approaches for morphometric grading of disease states. In particular, we conditioned graph convolutional networks on a population graph, whose nodes represented subject-specific shapes and whose edge weights encoded similarities between subjects. Based on this graph, the learning task could be formulated as a semi-supervised node classification problem, where labels are only given for nodes corresponding to subjects from the training set. The resulting grading systems could be successfully applied to the detection of Alzheimer’s disease from hippocampi shapes and the scoring of osteophytes in knee bones. We further adapted the transductive approach to derive a retrieval system for cultural heritage objects that learns object embeddings such that pair-wise distances encode task-specific similarities.
Existing graph neural networks adhere to the geometry of the input domain, that is, the graph; however, they assume that signals take values in vector spaces. In many cases – for example, when dealing with shape data – the signals belong to non-trivial, geometric spaces on their own, and it is only consequential to ask for models that use the geometry of the signal space as well. We constructed a novel graph convolutional layer based on a manifold-valued graph diffusion equation and derived node-wise multilayer perceptrons, both of which are equivariant not only with respect to node permutations but also isometries of the feature manifold. These filters are not only promising for manifold input signals but also allow for geometric representation learning. In particular, heterogeneous degree distributions and strong clustering in complex networks can often be explained by assuming an underlying hierarchy that is well captured in hyperbolic space. Indeed, our geometric filters enabled novel hyperbolic embedding strategies and learning of hierarchical structures that led to substantial improvements in full-graph classification. We further derived an equivariant temporal convolutional network that enables gesture recognition from skeletal data. These successes provide a strong impetus for further investigation of geometric deep learning techniques and create pathways for exploring their potential for open challenges in artificial intelligence and its applications.
Christoph v. Tycowicz
